Fix make secrets-init — SOPS matches creation rules on the input path - #6
Merged
Conversation
Reported from a real run on the monitoring host:
-- creating observability.sops.yaml from the template
error loading config: no matching creation rules found
make: *** [Makefile:62: secrets-init] Error 1
SOPS selects a creation_rule by matching path_regex against the *input* path.
The script encrypted the template and redirected the output:
sops --encrypt secrets/observability.example.yaml > secrets/observability.sops.yaml
so SOPS tested `secrets/.*\.sops\.ya?ml$` against the **example** filename,
which does not end in .sops.yaml. No rule matched. The output name is never
consulted, so the destination being correct made no difference.
Copy to the destination name first, then encrypt in place, which is the path the
rule actually targets.
Two guards added, because the failure mode of getting this wrong is worse than
an error message. `cp` followed by a failed encrypt leaves a plaintext file
sitting at a path whose name says "encrypted" — and it is not gitignored,
because encrypted secrets are meant to be committed. So on any encryption
failure the partial file is removed, and success is confirmed by checking for
the `sops:` metadata block rather than trusting the exit status. CI catches a
plaintext secrets file, but only after it has been pushed.
This was never caught because the whole path needs an age key, and the earlier
work stopped at "cannot verify without the user's key" rather than generating a
throwaway one.
Verified end to end in a sandbox with its own HOME and a disposable keypair:
1. bootstrap produces a genuinely encrypted file — keys readable, values
ENC[AES256_GCM...], no placeholder text remaining
2. `sops --encrypt --in-place` round-trips after editing values
3. render-config.sh writes snmp.yaml, webhook_url and .env
4. a community containing / & \ and $ renders byte-for-byte intact, and all
four devices keep distinct values
5. no SNMP community reaches .env — only the two values compose interpolates
6. `docker compose config` passes against the real rendered .env
7. git sees only .sops.yaml and the encrypted secrets file as stageable; no
.env, .rendered/ or plaintext copy can be committed
There was a problem hiding this comment.
Pull request overview
Updates the bootstrap flow that underpins make secrets-init so SOPS selects the correct creation_rule by ensuring the encrypted file’s input path matches .sops.yaml’s path_regex.
Changes:
- Copy the plaintext template to the destination
*.sops.yamlfilename first, then runsops --encrypt --in-placesocreation_rulesmatch the input path. - Add guardrails to delete the destination file if encryption fails or if the output does not contain the expected
sops:metadata.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+79
to
+96
| cp "${EXAMPLE_FILE}" "${SECRETS_FILE}" | ||
| chmod 600 "${SECRETS_FILE}" | ||
|
|
||
| if ! sops --encrypt --in-place "${SECRETS_FILE}"; then | ||
| rm -f "${SECRETS_FILE}" | ||
| die "encryption failed — removed the partial file rather than leave | ||
| plaintext credentials sitting at a path that looks encrypted. | ||
|
|
||
| Check that .sops.yaml lists a valid age recipient: | ||
| grep -A2 creation_rules ${SOPS_CONFIG}" | ||
| fi | ||
|
|
||
| # A plaintext file at this path would be committed as if it were encrypted, | ||
| # and CI only catches that after the push. Verify before claiming success. | ||
| if ! grep -q '^sops:' "${SECRETS_FILE}"; then | ||
| rm -f "${SECRETS_FILE}" | ||
| die "sops reported success but produced no encrypted output — file removed" | ||
| fi |
…ecks
Review feedback, and correct: cleanup only covered the two failures explicitly
checked. Between the `cp` and a verified encryption the file is plaintext at a
path that is meant to be committed and whose name says "encrypted" — and it is
deliberately not gitignored, so anything leaving it behind is a leak waiting to
be committed. A chmod failure under `set -e`, a Ctrl-C mid-encrypt, or a SIGTERM
all skipped the rm.
A trap on EXIT INT TERM is now armed immediately after the cp and cleared only
once the sops metadata block is confirmed present.
Verified in an isolated sandbox with its own HOME and a disposable keypair:
* happy path still produces an encrypted file
* a broken age recipient leaves nothing behind
* SIGINT delivered to the script's own process group mid-encrypt — a real
Ctrl-C — leaves nothing behind: present before the signal, absent after
Three earlier attempts at that last test were inconclusive rather than passing,
which is worth recording because each one looked like a result:
* signalling a process group that, in a non-interactive shell, also contained
the test harness
* sending SIGINT to the script's pid, where bash defers the handler until the
foreground command returns — sops completed and the file legitimately
survived encrypted, which reads as a leak unless you check the contents
* a setsid run whose $! had already exited, so the kill addressed an empty
pgid and no signal was ever sent
Only the setsid + pgrep-resolved pgid version actually exercises the path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hit on a real run on the monitoring host:
Cause
SOPS selects a
creation_ruleby matchingpath_regexagainst the input path. The script encrypted the template and redirected the output:sops --encrypt secrets/observability.example.yaml > secrets/observability.sops.yamlso SOPS tested
secrets/.*\.sops\.ya?ml$against the example filename, which doesn't end in.sops.yaml. No rule matched. The output name is never consulted, so having the destination right made no difference.Fix
Copy to the destination name first, then encrypt in place — the path the rule actually targets.
The plaintext window
Between the
cpand a verified encryption, that file is plaintext at a path meant to be committed, whose name says "encrypted" — and it is deliberately not gitignored, because encrypted secrets belong in git. Anything that leaves it behind is a leak waiting to be committed.The first version cleaned up only on the two failures it explicitly checked. Review correctly pointed out that a
chmodfailure underset -e, a Ctrl-C mid-encrypt, or a SIGTERM all skipped therm. A trap onEXIT INT TERMis now armed immediately after thecpand cleared only once thesops:metadata block is confirmed present. Success is verified by inspecting the file, not by trusting the exit status — CI does catch a plaintext secrets file, but only after it has been pushed.Why it wasn't caught earlier
This path needs an age key end to end, and the earlier work stopped at "cannot verify without the user's key" instead of generating a throwaway one. That was the wrong call: a disposable keypair in a sandbox
HOMEexercises the whole chain without touching anything real.Verification
Sandbox clone, its own
HOME, disposable keypair:ENC[AES256_GCM...], no placeholder text leftsops --encrypt --in-placeround-trips after editing valuesrender-config.shwritessnmp.yaml,webhook_urland.env/ & \ $renders byte-for-byte intact, and all four devices keep distinct values.env— only the two values compose interpolatesdocker compose configpasses against the real rendered.env.sops.yamland the encrypted secrets file as stageableThat is the first time the secrets path has been exercised end to end rather than reasoned about.
On test 9
Three earlier attempts were inconclusive rather than passing, and each looked like a result:
sopscompleted and the file legitimately survived encrypted, which reads as a leak unless you check the contentssetsidrun whose$!had already exited, so the kill addressed an empty pgid and no signal was ever sentOnly
setsidplus apgrep-resolved pgid actually exercises the path. Same failure mode as thexargsbug fixed in #5: a check that reports something without having tested what you think it tested.What to do on the host
The keypair and
.sops.yamlwritten by the failed run are correct — only the encrypt step failed, so nothing needs undoing:Then commit
.sops.yamlandsecrets/observability.sops.yamltogether, and carry on withmake validateandmake up.Back up
~/.config/sops/age/keys.txtoff that machine first. Without it, every encrypted secret in this repository is unrecoverable.